Skip to content

feat(spec): declare the aggregate × field-type compatibility matrix (AggregationFunction × FieldType) dataset measures are refused against (#16353) - #16684

Merged
os-trump merged 3 commits into
mainfrom
claude/issue-16353-aggregate-field-type-matrix
Sep 8, 2026
Merged

feat(spec): declare the aggregate × field-type compatibility matrix (AggregationFunction × FieldType) dataset measures are refused against (#16353)#16684
os-trump merged 3 commits into
mainfrom
claude/issue-16353-aggregate-field-type-matrix

Conversation

@os-trump

@os-trump os-trump commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Fixes #16353

Spec half of the split ruled on the analytics compile card (director seat, decision batch #59, 2026-09-06, "both legs, table in spec"). #16099 remains open — the compile-time refusal is that card's leg and is not addressed here; the authoring-time lint rule is its devx sub-card. Both legs consume the one table this PR declares.

Clause-②: yes — a previously accepted authoring shape (DatasetMeasureSchema accepts any aggregate × field pair today) becomes refused once the two consumer legs land against this table. The PR carries needs:contract-review.

Patch round (contract review VERDICT: FAIL, three findings adopted verbatim; no row changed): (1) the published ground for the boolean row was false and is replaced by what this tree can defend; (2) the predicate is now fail-closed on SHAPE as well as vocabulary; (3) the min / max refusal over the string classes is named as an override of measureResultType (#15768). The time justification is corrected for SQLite (canonical TEXT, #3994). Details in each section below.

What changed

  • New module packages/spec/src/data/aggregate-field-type-compatibility.ts, exported from @objectstack/spec/data:
    • AGGREGATE_FIELD_TYPE_COMPATIBILITY: Readonly Record over AggregationFunction of readonly FieldType arrays — frozen; total over AggregationFunction (the Record key type makes a missing row a tsc error) and total over FieldType (every member is in or out of every row).
    • isAggregateCompatibleWithFieldType(aggregate, fieldType) — the one predicate both legs call. Fail-closed on vocabulary (a driver alias such as integer, a retired array_agg, the ruling's countDistinct spelling all answer false) AND on shape: anything that is not a string answers false. The shape half is the patch-round fix — a property-key lookup alone applies ToPropertyKey, so ['count'] and an object whose toString returns 'sum' coerced to member spellings and passed a refusal gate; a typeof guard now precedes the lookup.
    • TSDoc states the rule, the reason (backend-divergent results, Prime Directive Add comprehensive test suite for Zod schema validation #12), how each ruling category was resolved against the real membership, the relation to isIncoherentAggregate, and — as of the patch round — the two rows that override existing opinions, recorded as overrides rather than as agreement.
  • Pin test aggregate-field-type-compatibility.test.ts (18 cases): row totality against AggregationFunction.options; classification totality against FieldType.options; the ruled rows pinned as literal sets; the numeric bucket held equal to NUMERIC_VALUE_TYPES and the temporal bucket to CALENDAR_DATE_TYPESINSTANT_TYPESCLOCK_TIME_TYPES; the motivating defect (avg × datetime) refused; the isIncoherentAggregate divergence pinned visibly; new: the shape guard pinned over arrays, toString objects, null, undefined, numbers and a symbol on both parameters, and the two recorded overrides pinned as the rows they are.
  • Barrel packages/spec/src/data/index.ts: one export * line.
  • Baselines regenerated by check:generated --fix after a full spec build: api-surface/data.json and export-origins/data.json each gain exactly the two new exports (4 added lines in total). The patch round changed no signature; check:generated on the rebuilt dist reports all 15 artifacts current.
  • Changeset .changeset/aggregate-field-type-compatibility.md, @objectstack/spec: minor — states the narrowing, why it is not breaking in this release, and (patch round) the two recorded overrides with their honest grounds.

The table (ruled rows, finalised against the full membership)

Measured on origin/main at d5c40222f3: FieldType has 49 members, AggregationFunction has 6 (count, sum, avg, min, max, count_distinct).

Aggregate Accepted Count
count, count_distinct every FieldType 49
sum number, currency, rating, slider, progress, summary — numeric class EXCEPT percent 6
avg number, currency, percent, rating, slider, progress, summary 7
min, max the 7 numeric plus date, datetime, time 10

How the ruling's categories resolved (the part the card delegated to the spec seat):

  • "numeric types"NUMERIC_VALUE_TYPES in field-value.zod.ts (number, currency, percent, rating, slider, progress, summary) — the class the SQL DDL stores as REAL columns (sql-driver.ts createColumn).
  • "integer-class" → no FieldType member is named integer; integer / int are driver-internal column aliases (type-compat.ts, sql-driver.ts). The integer-valued authorable members are rating, slider, progress, which land in the numeric bucket above.
  • "temporal (date, datetime)" → the ruling named two; time is the third temporal class (CLOCK_TIME_TYPES) and takes the same treatment. Corrected justification: its stored form is a dialect question exactly like the other two — native TIME on Postgres, TIME(3) on MySQL, canonical HH:MM:SS[.fff] TEXT on SQLite (Field.time repeats the #3912 pattern: writes unnormalised, repaired only on read — window filters and ORDER BY are silently wrong on SQLite #3994) — the canonical form orders chronologically on every one of them, and AnalyticsResult.fields[].type already describes min / max over it as temporal (A dataset measure over a datetime is typed number in the analytics response, and a metric tile ignores the format style it is handed — measured on 17.3.0 #15768, TEMPORAL_SOURCE_FIELD_TYPES lists time). Placed with date / datetime for min / max only; reported as a completion the ruling did not spell out.
  • Refused for sum / avg / min / max (39 members): address, audio, autonumber, avatar, boolean, checkboxes, code, color, composite, email, file, formula, html, image, json, location, lookup, markdown, master_detail, multiselect, password, phone, qrcode, radio, record, repeater, richtext, secret, select, signature, tags, text, textarea, toggle, tree, url, user, vector, video. formula is VIRTUAL in SQL storage (createColumn emits no column), so no arithmetic aggregate can be lowered to it whatever its declared returnType.

Two refused rows that override existing opinions (recorded, not resolved; rows unchanged)

  1. Booleans (boolean, toggle) — patch-round correction. The previous text called them "the divergence class" (one dialect sums 0/1, another has no sum(boolean)). That is false at this tree: maintainer ruling [finding] AGGREGATION_ROWS has no boolean column, so the cross-driver aggregation conformance family cannot see a boolean aggregand on any face #11152 pins that booleans aggregate as numbers on every face with no per-aggregate exception — AGGREGATION_CASES in packages/spec/src/data/aggregation-conformance.ts pins sum(flag)=3, avg(flag)=0.5, min(flag)=0, max(flag)=1 across six backends — and driver-sql casts a boolean aggregand to int on Postgres to make that hold (sql-driver.ts, driver-sql: boolean aggregands need a lowering cast on PG (+ a MySQL min/max presentation check) — the ruled false/true + arithmetic answers are unproducible on the PG face #11635). The refusal is therefore NOT grounded in backend divergence; the backends agree. The row stays exactly as batch Validation Protocol: Cross-Field, Async, and Conditional validation #59 stated it ("every other pair: refused"). Whether booleans belong in sum / avg / min / max is a collision between two maintainer rulings (batch Validation Protocol: Cross-Field, Async, and Conditional validation #59 vs [finding] AGGREGATION_ROWS has no boolean column, so the cross-driver aggregation conformance family cannot see a boolean aggregand on any face #11152) and goes to the maintainer as its own decision card; the TSDoc and changeset now say so instead of the false claim.
  2. The string classes for min / max — undeclared contradiction, now declared. service-analytics' measureResultType (A dataset measure over a datetime is typed number in the analytics response, and a metric tile ignores the format style it is handed — measured on 17.3.0 #15768, merged) types min / max over STRING_SOURCE_FIELD_TYPES (STRING_VALUE_TYPESSINGLE_OPTION_TYPESREFERENCE_VALUE_TYPESautonumber) as a supported 'string' result. This table refuses those pairs. The refusal is defensible — string ORDER is collation-dependent, so two backends can return two different "smallest" values — but it overrides an existing runtime opinion, which the first round's sweep missed. Row unchanged; named here, in the TSDoc, in the changeset and in the report.

Conflicts and forks reported rather than resolved silently (unchanged from round 1; all judged defensible at review)

  1. count_distinct × percent. The existing spec opinion is isIncoherentAggregate in packages/spec/src/data/aggregation-policy.ts (the card's analytics-service.ts attribution was corrected on the card by the PM), and it flags this pair; the ruling reads count_distinct as "any type". The table follows the ruling; the divergence is pinned. Open question 1.
  2. time in min / max — completion beyond the two members the ruling named. Open question 2.
  3. progress stays in sum by the mechanical rule (the ruling excluded only percent, on isIncoherentAggregate's authority). Open question 3.
  4. formula with returnType: 'number' — refused (virtual in SQL storage; a FieldType-keyed table cannot see returnType). Open question 4.

Why the changeset is minor, not breaking

This PR ships a table and a predicate that nothing yet enforces: DatasetMeasureSchema accepts byte-for-byte what it accepted before, no export is removed or narrowed, no runtime path reads the table. Two new exports in the published .d.ts are an additive widening of the public surface (minor per the 2026-09-04 ruling on #15294). The refusal — and the breaking declaration, migration prescription and ADR-0087 disposition — arrives with the consumer legs' changesets. The changeset text states the narrowing so an upgrading author reads the contract before it is executed.

Verification — patch round (all at d23f7705b8, the final commit; tree clean)

  • Build: turbo run build --concurrency=2 --filter=@objectstack/formula --filter=@objectstack/lint under os-verify-lock.sh (builds @objectstack/spec as their dependency with its full build script: gen:schema + gen:openapi + tsup + DTS + check-dts-emitted + check-dev-prereqs --stamp) — VERDICT command-exit 0, 4 tasks successful; gen:schema moved no tracked file.
  • Tests: vitest run src/data/aggregate-field-type-compatibility.test.ts src/data/aggregation-policy.test.ts src/data/field-value.test.tsTest Files 3 passed (3) · Tests 49 passed (49) (18 in the new file).
  • Typecheck: pnpm --filter @objectstack/spec typecheckVERDICT command-exit 0; test-layer ledger unchanged (54 file(s) / 261 error(s) / 145 pinned signature(s) held).
  • Generated artifacts: check:generated on the rebuilt dist — ✓ All 15 generated artifacts are up to date. (no signature moved; no regeneration needed this round).
  • Ablation of the new guard, from the committed state: removed the typeof line (anchor count 1→0, marker 0→1 on disk; blob c7b31303…cb2a65c7…) → Tests 1 failed | 17 passed (18): exactly the shape pin red, everything else green. Restored with git checkout HEAD -- ABSOLUTE_PATH under an EXIT/INT/TERM trap; proof: git diff HEAD empty, git status --porcelain empty, disk blob c7b31303… equals the HEAD blob, marker count 0. Round 1's row ablation (percent into sum → 3 pins red, byte-identical restore) stands.
  • CJS probe through the published require entries (rebuilt dist): all four entries load; both ./data entries answer sum×percent false, min×time true, and — through the published build — ['count']×number false and an object whose toString returns 'sum' × currency false.
  • Gate families derived by node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands (no paths; from the merge base at d23f7705b8): 76 commands, identical to round 1's list, all run, exact strings recorded, --ran verdict ✓ 76 derived famil(ies) accounted for — 76 run, 0 NOT-MEASURED. Exit codes: 74 × 0 (line 31 check:doc-formula-expressions now 0 directly, its prerequisite built in the same turbo run); two × gate-declared 3 (PREREQUISITE NOT MET — the gate's own code, not a shell timeout; nothing measured, not a finding):
    • line 55 pnpm check:dual-build-cjs-loads — needs every package's dist/ (whole-farm build; no package-scoped mode). Declared narrowing: the CJS probe above over the only package the diff touches. CI runs the full gate.
    • line 73 pnpm check:type-check-debt — the re-measure needs the whole package closure built. Declared narrowing: spec's own typecheck green (above); check:type-check-coverage (line 72) exit 0. CI runs the full gate.
  • pnpm check:nul-bytes exit 0 (line 59).

验收备注

维护者速读(草稿)

改了什么:在 @objectstack/spec/data 里新增一张「聚合函数 × 字段类型」兼容表和一个判定函数。表按董事裁决(第 59 批)填好六行:count / count_distinct 对全部 49 种字段类型放行;sum 只对可相加的数值类型放行(不含百分比);avg 对含百分比的数值类型放行;min / max 再加上日期、日期时间、时间三种。其余搭配一律记为拒绝。本 PR 只声明这张表,不做任何拒绝。补丁轮:判定函数对非字符串输入(数组、带 toString 的对象)一律拒绝;文档不再声称「布尔值是后端分歧类」——现有裁决 #11152 已让所有后端把布尔值当数字聚合——改为如实记录该行沿用裁决、成员归属另立决策卡;min / max 拒绝字符串类字段一事,注明它覆盖了 #15768 已有的运行时意见。

为什么改:今天一个数据集度量可以把「平均值」套在日期时间字段上,编译成 AVG(col) 直达数据库——SQLite 返回一个毫秒平均数,Postgres 直接报错。同一份元数据在两个后端给出两个答案,正是 Prime Directive #12 要消灭的形态。哪些搭配可接受是一份契约,应当在 spec 声明一次,由编译端(#16099)和 lint 端两条腿共同执行。

风险与代价(含回滚):本 PR 本身不改变任何现有行为——schema 接受的内容逐字节不变,没有运行时读取这张表。真正的收窄随两条消费腿落地,届时今天能写的部分度量会被拒绝,需要作者把聚合改为字段类型支持的那一个。回滚只需删除这个模块、一行 barrel 导出和两处基线条目。需要维护者留意的判断:① 布尔值是否应进入 sum / avg / min / max——第 59 批裁决(默认拒绝)与 #11152(所有后端按数字作答)相撞,表暂按第 59 批,问题另立决策卡;② min / max 拒绝字符串类字段,而 #15768 已把它们的结果类型定为字符串——拒绝有理(排序依赖 collation),但覆盖了既有意见;③ 现有 isIncoherentAggregate 认为 count_distinct × percent 不合理,而裁决说 count_distinct 对任意类型放行;④ 裁决只点名 date / datetime,本表把 time 一并放进 min / max;⑤ progress 留在 sum 里。

席位意见:(留空,由席位定稿)

你要做的:确认上面五个判断是否符合预期,尤其是①的布尔值归属决策;若认可,本 PR 作为两条消费腿的前置合入。合并不关闭 #16099

🤖 Generated with Claude Code — session https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf

…16353)

Export AGGREGATE_FIELD_TYPE_COMPATIBILITY and isAggregateCompatibleWithFieldType
from @objectstack/spec/data: the one table the dataset compiler and the lint
rule refuse dataset measures against. Rows follow the director ruling
(decision batch #59), resolved against the full FieldType membership through
the field-value semantic classes; pinned literally in the test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf
…and export-origins baselines (#16353)

Regenerated by `check:generated --fix` after a full spec build: the two
stale shards (api-surface/data.json, export-origins/data.json) each gain the
two new exports and nothing else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation protocol:data tests tooling labels Sep 7, 2026
@os-trump os-trump added needs:contract-review and removed documentation Improvements or additions to documentation tests tooling protocol:data labels Sep 7, 2026 — with Claude
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec, touching 7 documentable anchor(s). ⚠️ 3 changed file(s) yielded no anchor (packages/spec/api-surface/data.json, packages/spec/export-origins/data.json, packages/spec/src/data/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

5 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/ai/natural-language-queries.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))
  • content/docs/data-modeling/queries.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))
  • content/docs/kernel/contracts/data-engine.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))
  • content/docs/protocol/objectql/query-syntax.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))
  • content/docs/ui/dashboards.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))

2 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v15.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))
  • content/docs/releases/v17.mdx (via count_distinct (symbol, a field of const object AGGREGATE_FIELD_TYPE_COMPATIBILITY))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 3 changed file(s) yielded no anchor (packages/spec/api-surface/data.json, packages/spec/export-origins/data.json, packages/spec/src/data/index.ts) — pages documenting those are invisible to this run
  • 4 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 130 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json c8e5ac645f65dbaed0e44d8e40644f28a156a022packageMentionDocs.

Which tree this was computed on

This run read content/docs from d232adddc49a668e2b152759b0e5b39fdcc043e7 — the merge of head d23f7705b83a259bfd192b178889a201564ee997 into base c8e5ac645f65dbaed0e44d8e40644f28a156a022, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin d232adddc49a668e2b152759b0e5b39fdcc043e7 && git checkout d232adddc49a668e2b152759b0e5b39fdcc043e7
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin c8e5ac645f65dbaed0e44d8e40644f28a156a022 d23f7705b83a259bfd192b178889a201564ee997 && git checkout -B drift-repro c8e5ac645f65dbaed0e44d8e40644f28a156a022 && git merge --no-ff d23f7705b83a259bfd192b178889a201564ee997

node scripts/docs-audit/affected-docs.mjs --json c8e5ac645f65dbaed0e44d8e40644f28a156a022

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs c8e5ac645f65dbaed0e44d8e40644f28a156a022 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

…ype; correct the published grounds for the boolean and time rows (#16353)

Contract-review patch round. The predicate now refuses any non-string input
(a property-key lookup alone coerced ['count'] / { toString } to a member
spelling); pinned. The TSDoc and changeset no longer claim booleans are the
divergence class - #11152 has every backend answer them as numbers - and
record that row, plus the min/max refusal over the string classes (#15768
types them as a supported 'string' result), as overrides of existing
opinions referred to the maintainer. The time justification names SQLite's
canonical TEXT form (#3994). No row changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/m tests tooling

Projects

None yet

2 participants